Target-less auto-alignment of image sensors in a multi-camera system

ABSTRACT

A pair of cameras having an overlapping field of view is aligned based on images captured by image sensors of the pair of cameras. A pixel shift is identified between the images. Based on the identified pixel shift, a calibration is applied to one or both of the pair of cameras. To determine the pixel shift, the camera applies correlation methods including edge matching. Calibrating the pair of cameras may include adjusting a read window on an image sensor. The pixel shift can also be used to determine a time lag, which can be used to synchronize subsequent image captures.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/345,002, filed Nov. 7, 2016, now U.S. Pat. No. 9,792,667, which is acontinuation of U.S. patent application Ser. No. 14/876,041, filed Oct.6, 2015, now U.S. Pat. No. 9,521,318, which is a continuation of U.S.patent application Ser. No. 14/221,256, filed Mar. 20, 2014, now U.S.Pat. No. 9,197,885, all of which are hereby incorporated by reference intheir entirety.

BACKGROUND 1. Field of Art

The disclosure generally relates to the field of digital cameras, andmore particularly to capturing synchronized images from multiplecameras.

2. Description of the Related Art

Digital cameras capture images using an electronic image sensor. Exampleimage sensors use CCD (charge-coupled device) or CMOS (complementarymetal-oxide semiconductor) technology. A CMOS image sensor containsphotodiodes corresponding to pixels of a captured image. In oneconfiguration, the photodiodes are arranged in rows and columns. Tocapture an image, successive rows of photodiodes are exposed and readover time. The direction in which rows of photodiodes are exposed overtime is the rolling shutter direction. For example, if an image sensorreads a row of photodiodes after reading the row of photodiodes aboveit, then the image sensor has a downward rolling shutter direction.Portions of the image along the rolling shutter direction are exposed atlater times. When capturing an image of a scene in which an object ismoving or when capturing an image in which the camera is moving relativeto one or more objects within the image, objects may appear blurred ifthe object moves in an image frame relative to different image sensorrows exposed at different times. Similarly, duplicates of an object mayappear (e.g., an image of a spinning airplane propeller captured with aCMOS sensor). Such image artifacts are referred to as “rolling shutterartifacts.”

Three-dimensional (3D) images demonstrate depth information in acaptured scene. One method for forming 3D images involves thecombination of two or more two-dimensional (2D) images captured fromdifferent perspectives. An object's depth may be inferred based on theobject's positions within the 2D images and based on the relativeperspectives from which the 2D images were captured. When capturing amoving scene, the 2D images are taken simultaneously. Otherwise, captureobjects moving relative to an image frame can appear at an incorrectdepth in the resulting 3D image. When the rolling shutters of twocameras are misaligned, capturing the 2D images even simultaneously canresult in rolling shutter artifacts and incorrect object depthdeterminations, as different portions of a scene are captured atdifferent times. If the cameras do not capture the same portion of thescene at the same time, then any moving elements of the scene may appearblurred. To synchronize image capture, cameras containing the imagesensors can be aligned along the direction of the rolling shutter usinga six-axis pan-tilt camera mount, but aligning the cameras by thismethod is a time-consuming task, even for an experienced photographer.

BRIEF DESCRIPTION OF DRAWINGS

The disclosed embodiments have other advantages and features which willbe more readily apparent from the detailed description, the appendedclaims, and the accompanying figures (or drawings). A brief introductionof the figures is below.

FIG. 1A illustrates an example pair of cameras configured withoverlapping fields of view, according to one embodiment.

FIGS. 1B and 1C illustrate two cameras arranged in exampleconfigurations to capture an image, according to one embodiment.

FIG. 1D illustrates example images taken by a pair of cameras withmisaligned rolling shutters, according to one embodiment

FIG. 2 illustrates an example architecture for each of a pair ofcameras, according to one embodiment.

FIG. 3 illustrates an example high-level block diagram of a system forsynchronizing a pair of image sensors, according to one embodiment.

FIG. 4A illustrates the alignment of an example pair of image sensors byshifting the read window on an image sensor of the pair, according toone embodiment.

FIG. 4B illustrates the alignment of an example pair of image sensors byapplying a time lag, according to one embodiment.

FIGS. 5A-5D illustrate example images illustrating edge detection usedfor determining a pixel shift, according to one embodiment.

FIG. 6A illustrates a flow diagram of a process for aligning imagesensors in a multi-camera system, according to one embodiment.

FIG. 6B illustrates a flow diagram of a process for determining a pixelshift between images having an overlapping field of view, according toone embodiment.

DETAILED DESCRIPTION

The Figures (FIGs.) and the following description relate to preferredembodiments by way of illustration only. It should be noted that fromthe following discussion, alternative embodiments of the structures andmethods disclosed herein will be readily recognized as viablealternatives that may be employed without departing from the principlesof what is claimed.

Reference will now be made in detail to several embodiments, examples ofwhich are illustrated in the accompanying figures. It is noted thatwherever practicable similar or like reference numbers may be used inthe figures and may indicate similar or like functionality. The figuresdepict embodiments of the disclosed system (or method) for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles described herein.

Configuration Overview

In one embodiment, a computer-implemented process synchronizes a pair ofimage sensors. Images are captured with the pair of image sensors atsubstantially the same time. The image sensors have an overlapping fieldof view. A pixel shift is identified between the captured images basedon captured image data representative of the overlapping field of view.A time lag is determined between the image sensors based on theidentified pixel shift. The pair of image sensors is calibrated based onthe determined time lag or the identified pixel shift to synchronizesubsequent image capture by the image sensors.

In one embodiment, a computer-implemented process determines a pixelshift between a pair of images. A pair of images captured atsubstantially the same time is obtained. The images comprise image datarepresentative of an overlapping field of view. One or more edges in theimage data are identified. The one or more edges correspond to edges ina first image and a second image of the pair of images. The one or moreidentified edges from the first image are matched to the one or moreidentified edges in the second image. The pixel shift is determinedbetween the pair of images based, at least in part, on the matchingbetween the one or more edges.

It should be noted that although the examples described herein includethe alignment of image sensors between two cameras, in practice, theprinciples described herein can apply to an array of any number ofcameras. For instance, an average pixel shift or read window shift canbe determined and applied to a linear array of three or more cameras inorder to align the array of cameras. Further, a pixel shift or readwindow shift can be determined for both horizontally adjacent cameras,or for vertically adjacent cameras. For example, in a 2×2 camera array,images sensors can be aligned by determining a horizontal pixel shift orread window shift between a top pair of cameras and a bottom pair ofcameras in the array, and by determining a vertical pixel shift or readwindow shift between a left pair of cameras and a right pair of camerasin the array.

Image Capture with a Pair of Cameras

FIG. 1A illustrates an example pair of cameras configured withoverlapping fields of view, according to one embodiment. In oneembodiment, the adjacent cameras 100A and 100B belong to a camera pairand are configured to capture synchronized image data. The image datafrom the camera pair may be stitched together to form panoramic imagesor video, which may be two-dimensional (2D) or three-dimensional (3D).

As shown, each camera 100 includes a lens assembly 102, a shutter button104, a function button 106, an indicator light 108, a control displayscreen 110, and a memory card slot 112 contained in a camera body. Thebody of the camera secures the components of camera. The lens assembly102 includes one or more lenses and/or other optical components designedto focus, direct, converge and/or diverge light onto an internal imagesensor (not illustrated in the embodiment of FIG. 1A). The lens assemblymay include electrical and/or mechanical components to change the lensassembly's configuration (e.g., to modify the camera's focus and/orfield of view).

Other conventional camera features not specifically illustrated ordescribed may also be included in one or both cameras. For example, acamera 100 may include various optical components, an image sensor(e.g., a complementary metal oxide semiconductor (CMOS) sensor), amicrophone, speakers, an image and/or audio processor, a controller,internal storage, an image preview display screen, input/output ports, acommunications module, and/or other conventional components. A cameramay include one or more components to secure the camera to a mount. Inone embodiment, the cameras 100 are secured to a common mount. The mountmay include a housing to encompass all or part of the cameras 100 invarious positions and/or orientations.

In an alternative embodiment, the adjacent cameras 100A and 100B arecombined into a single camera body having at least two lenses 102 andother components for capturing image data from at least two distinctperspectives. The cameras 100 may include different or additionalcontrol inputs than those illustrated in FIG. 1A for capturing imagesand/or videos, for viewing previously captured images and/or videos, orfor managing various camera settings. The illustrated components may beomitted and/or modified without substantially inhibiting thefunctionality described herein.

Each camera 100 may have one of a variety of form factors. In oneembodiment, each individual camera 100 in the camera system has a smallform factor (e.g., a height of approximately 4 to 6 centimeters, a widthof approximately 5 to 7 centimeters, and a depth of approximately 2 to 4centimeters), and is lightweight (e.g., approximately 50 to 150 grams).Alternately or additionally, each camera 100 may have any suitable formfactor configured to enable the functionalities as described herein.

In one embodiment, the cameras 100 may function independently. Whenoptionally synchronized, multiple cameras can simultaneously capturevideo or still images in a time-synchronized manner. In one embodiment,the cameras communicate using a master/slave (or primary/secondary)configuration in which one camera operates as the master and one moreadditional cameras operate as slaves. In this configuration, the mastercamera sends various commands to the slave cameras. For example, themaster camera may send a command that instructs the slave cameras tocapture an image or begin recording video at a particular time (e.g., apredefined instance) or at a specific event (e.g., a specific eventwithin the camera such as capture of a first frame of video or otherimage). Furthermore, the master camera may configure various settings ofthe slave cameras such as, for example, frame rate, exposure time,resolution, color, and/or any other operating parameters to ensure thatthe cameras record video or images using the same setting.

In one embodiment, either camera can be configured as either a master orslave (e.g., using a physical switch or a software setting). For videoor image capture, the cameras 100 are time-synchronized to ensure thateach of the synchronized cameras initializes frame capture at the sametime. Furthermore, the cameras' clocks can be periodicallyre-synchronized in order to compensate for potential drift due todifferent internal camera clock characteristics. In one embodiment,captured images and/or videos are transmitted to a remote device forviewing. Alternatively or additionally, captured images and/or videostransmitted for display on one or more of the cameras 100 (e.g.,displayed on a screen integrated with the camera body). A camera 100 mayrecord and store its data locally, on a paired camera, and/or on anexternal device.

As shown in FIG. 1A, the cameras 100 can be similarly positioned suchthat the surfaces of their lenses 102 and top surfaces are substantiallycoplanar such that the cameras' line of sight is substantially parallel.In other example embodiments, the cameras 100 are rotated (e.g., towardsor away from each other) in the plane formed by their top or sidesurfaces, such that the lens assembly 102 surfaces are not coplanar. Thedistance between lenses 102A and 102B may also differ depending on thedesired application.

Each lens assembly 102 includes a corresponding field of view (“FOV”)which denotes the physical space from which a camera 100 may captureimage data. Although the FOVs are similar in size as illustrated in FIG.1A, different FOVs may result when the lenses 102 or zoom settingsdiffer between the cameras. The edges of camera 100A's FOV are denotedby FOV boundary lines 115A and 116A. Also shown is the rolling shutterdirection 117A of camera 100A (from boundary line 115A to boundary line116A). The edges of camera 100B's FOV (boundary lines 115B and 116B) aresimilarly shown along with its rolling shutter direction 117B.

As a result of the positions and orientations of cameras 100A and 100B,the fields of view overlap to form an overlap region 114, which ispartially bounded by boundary lines 115A and 116B. As described above,if the cameras 100 are rotated towards each other or moved closertogether, the FOV overlap region 114 increases, and if the cameras 100are rotated away from each other or moved further apart, the overlapregion 114 decreases. Image distortion at FOV boundaries 115 and 116 canresult in various impracticalities for stitching collected image data.In practice, the actual size of the overlap region 114 may be alteredbased on the user's desired application or equipment qualities (e.g.,distortion from the lens assembly 102, the alignment of the camerahousing, the camera body, the lens assembly 102, and the image sensor).In either instance, rolling shutter artifacts may occur within theoverlap region 114.

To demonstrate an example circumstance producing a rolling shutterartifact, FIG. 1A illustrates an object 125 (e.g., a baseball) movingalong a path within the overlap region 114 during image capture. Theobject 125 is shown at a first position 125A and second position 125B.The first position 125A corresponds to a first time, t1, when therolling shutter of camera 100A captures the object 125. The secondposition 125B corresponds to a second time, t2, when the camera 100Bcaptures the object 125. As shown, the rolling shutter 117A of camera100A encounters the object 125 at an earlier position and time than therolling shutter of camera 100B. In other words, camera 100A “sees” orcaptures an image of the object 125 before camera 100B. Thus, cameras100A and 100B capture image data of the object 120 at two differentpositions. If the cameras were intended to capture an image insynchronization, then the images contain a rolling shutter artifact,which is the discrepancy in the position of the object 125.

Oftentimes, these rolling shutter artifacts create significant problemsduring post-processing. For example, when images from the cameras 100Aand 100B are combined to stitched together to create a 2D imageencompassing the FOVs of both cameras, the object 125 is shown indifferent positions. The combined image may show two instances of object125, which may overlap with each other depending on the magnitude of theposition discrepancy. The multiple instances of object 125 may interferewith aligning the respective FOVs of cameras 100A and 100B as well.

As another example, the images from cameras 100A and 100B are combinedto create a 3D image encompassing the overlap region 114. Cameras 100Aand 100B capture images from different perspectives, so the position ofobject 125 appears different between the two images due to parallax.This shift due to parallax may be used to infer the depth of the object125 from a plane containing the lenses 102. The depth of the object 125is inversely proportional to the apparent positional shift of thatobject (parallel to the axis between the lens apertures of cameras 100Aand 100B). Parallax assumes that the object 125 is at the same positionin both images. If object 125 has moved along the axis parallel to theline between the lens apertures, then the object's apparent shift in theimage data contains a shift component due to the object's movement and ashift component due to parallax. Due to the shift component from theobject's movement, the apparent depth calculated from the apparent shiftof the object 125 is inaccurate. 3D images resulting from imagescontaining a moving object captured at different times thus can containobjects at incorrect depths.

The cameras illustrated show one example configuration for producing 2Dand/or 3D images using image data from multiple perspectives. Otherexample configurations of cameras include cameras placed around acentral point and pointed substantially opposite the central point(e.g., four cameras aligned to provide a 360-degree field of view aroundan axis, six cameras aligned to provide a spherical field of view, andthe like). The image sensors may be arranged linearly or as part of anarray (e.g., an array of four coplanar cameras arranged in two rows andtwo columns). The cameras may be distributed and pointed towards acentral point. The disclosed techniques for aligning image sensors maybe used for any pair of cameras with an at least partially overlappingfield of view.

Alignment Along the Rolling Shutter Direction

Turning to FIGS. 1B and 1C, illustrated are two cameras arranged inexample configurations to capture an image according to one embodiment.Two cameras 100 are illustrated showing a lens assembly 102 and aninternal image sensor 126. Camera 100A is positioned above camera 100B,and the cameras are configured to capture images of the object 125,which is moving downwards. The cameras 100, object 125, and the distancebetween them are not shown to scale. The lines of sight 132 demonstratethe path taken by light from the object 125 to the image sensors 126.When a camera 100 captures an image, light from the object 125 entersthe camera through the lens assembly 102 and falls incident upon on theimage sensor 126. The object's apparent position in the captured imagedepends on the location of the image sensor 126 on which the light fromthe object 125 is incident, as shown by line of sight 132.

As manufactured or assembled, the relative positions and/or orientationsof a first camera's body and lens assembly within the camera body maydiffer from the relative positions and/or orientations of a secondcamera's body and lens assembly within the camera body, even if thecameras are manufactured to be identical. The example camera 100A, forexample, has a smaller distance between the lens assembly 102A and imagesensor 126A than camera 100B has between the lens assembly 102B andimage sensor 126B. These differences in position and/or orientation maychange depending on the location of the image sensor 126 on which theline of sight 132 is incident. Accordingly, these differences may changean object's apparent position within a captured image. Additionally, theassembly and/or manufacture of the components of the lens assembly 102may cause additional deviations in the line of sight between the imagesensor 126 and the object 125. For example, the relative position ofcomponents in the optical assembly, the alignment of a lens mountrelative to the optical assembly, the alignment of the lens mountrelative to the camera body, the alignment of the image sensor 126relative to the camera body, or the alignment of the camera bodyrelative to an attached camera mount may contribute to misalignment.These deviations in alignment may change the object's apparent positionbetween a pair of captured images.

In FIG. 1B, cameras 100A and 100B are configured so that the lines ofsight 132 result in different object positions on the respective imagesensors 126. This occurs, at least in part, because of the differentdistances between the respective lens assemblies 102 and image sensors126. FIG. 1D illustrates example images 140 taken by the cameras 100with misaligned rolling shutters, consistent with the embodiment of FIG.1B. Images 140A and 140B correspond to the images taken by cameras 100Aand 100B, respectively, in the configuration illustrated in FIG. 1B. Theimages contain pixel rows 143 and 146. In image 140A, object 125 iscaptured at pixel row 143A, and in image 140B, object 125 is captured atpixel row 146B. In an example image having 3000 pixels in the verticaldirection, the shift between pixel row 143 and 146 is about 100 pixels.This rolling shutter artifact is due to the difference in lens positionsbetween cameras 100A and 100B. In FIG. 1C, cameras 100A and 100B areconfigured so that the lines of sight 132 result in substantiallysimilar object positions on the image sensors 126 even though the lensassemblies 102 have different positions from the respective imagessensors 126.

Misalignments along the direction of the rolling shutter cause rollingshutter artifacts when capturing a moving object. For example, supposethe cameras 100 have a downward rolling shutter direction and begincapturing the image at the same time. The rolling shutter captures pixelrow 143 at an earlier time than it captures pixel row 146. Since theobject 125 is moving, images 140A and 140B capture the object atdifferent positions in space. If images 140A and 140B are combined, theresulting image will have a rolling shutter artifact.

Camera Architecture

FIG. 2 illustrates an example architecture for each of a pair of cameras100A and 100B, according to one embodiment. Camera 100A includes animage sensor 206, a sensor controller 208, a processor 202, a memory210, and a synchronization interface 220. Camera 100A is communicativelycoupled to camera 100B over network 230. In one embodiment, camera 100Bcontains similar components to camera 100B. Although two cameras 100 areillustrated, additional cameras may be present. Alternatively oradditionally, the cameras 100 are communicatively coupled over of thenetwork 230 to a computing device (not illustrated in FIG. 2) thatperforms at least some functions of the processor 202 and/or memory 210.Alternatively or additionally, the components of cameras 100A and 100Bare combined in a common camera body. For example, the combined cameramay have two or more image sensors 206 that share a common processor202, memory 210, or synchronization interface 220. It should be notedthat in other embodiments, the cameras 100 can include additional ordifferent components than those illustrated in the embodiment of FIG. 2.

The image sensor 206 is a device capable of electronically capturinglight incident on the image sensor 206. In one embodiment, CMOS sensorsare used, including transistors, photodiodes, amplifiers,analog-to-digital converters, and power supplies. In one embodiment, theimage sensor 206 has a rolling shutter direction. The image sensor 206can capture the portions of the image at substantially at the same time(e.g., using a CCD sensor). In one embodiment, the image sensor 206 hasan adjustable read window. An adjustable read window modifies theportions of the image sensor that are exposed to light and read tocapture an image. By adjusting the read window, the camera 100A canmodify when a portion of an image is captured relative to when imagecapture begins. For example, by shifting the read window in the rollingshutter direction, the image sensor captures portions of the image inthe read window earlier than if the read window was not shifted in therolling shutter direction.

The processor 202 is one or more hardware devices (e.g., a centralprocessing unit (CPU), a graphics processing unit (GPU), a digitalsignal processor (DSP), and the like) that execute computer-readableinstructions stored in the memory 210. The processor 202 controls othercomponents of the camera based on the instructions that are executed.For example, the processor 202 may send electronic control signals tothe image sensor 206 or use the synchronization interface 220 to senddata over the network 230 to camera 100B or some other connected device.

The memory 210 is a non-transitory storage medium that can be read bythe processor 208. The memory 210 may contain volatile memory (e.g.,random access memory (RAM)), non-volatile memory (e.g., a flash memory,hard disk, and the like), or a combination thereof. The memory 210 maystore image data captured by the image sensor 206 and computer-readableinstructions to be executed by the processor 202.

The sensor controller 208 controls operation of the image sensor 206 andother functions of the camera 100A. The sensor controller 208 caninclude physical and/or electronic input devices, such as exteriorbuttons to start recording video and/or capture a still image, atouchscreen with tap-to-focus capabilities, and a dial/buttoncombination for navigating a menu hierarchy of the camera 202. Inaddition, the sensor controller 208 may include remote user inputdevices, such as remote controls that wirelessly communicate with thecameras 100A. The image sensor 206 may function independently of thesensor controller 208. For example, a slave camera in a master-slavepairing can receive a signal to capture an image through thesynchronization interface 220.

The synchronization interface 220 sends and receives data over thenetwork 230 to communicate with other devices, such as camera 100B,other cameras, or an external computing system. In particular, thesynchronization interface 220 may send or receive commands to one ormore cameras for simultaneously capturing an image and/or calibratingsynchronization with other cameras (e.g., sending or receiving asynchronization pulse). The synchronization interface 220 may be used totransfer image data to another camera system for use in calibration.

The network 230 communicatively couples the cameras 100 and othercomponents not illustrated in the embodiment of FIG. 2, and allowscoupled devices, including the cameras 100, to exchange data with eachother over local-area or wide-area networks. The network 230 may containa combination of wired or wireless technology and make use of variousconnection standards and protocols, such as USB (universal serial bus),IEEE 1394, Ethernet, 802.11, or Bluetooth. The network 230 may also usethe internet to exchange data between connected devices and may containlink technologies such as CDMA (code division multiple access), 3G, 4G,or another wireless local-area network. Alternatively or additionally,the network 230 and the network synchronization interface are omittedand image and/or calibration data is transferred between the cameras 100using a non-transitory computer-readable medium (e.g., a removable flashmemory card).

Image Sensor Synchronization

FIG. 3 illustrates an example high-level block diagram of a system forsynchronizing a pair of image sensors, according to one embodiment. Thesystem includes modules stored on memory 210 including an image store305, a synchronization store 310, a capture controller 315, a pixelshift determination module 320, and a time lag determination module 330.Alternate embodiments may have one or more additional, omitted, oralternative modules configured to perform similar functionality. Itshould be noted that in other embodiments, the modules described hereincan be implemented in hardware, firmware, or a combination of hardware,firmware, and software. In addition, in some embodiments, a first camerain a pair includes the components illustrated in FIG. 3, while a secondcamera does not necessarily include the components of FIG. 3, butinstead merely synchronously captures an image with the first camera andprovides the captured image to the first camera for processing. As usedherein, a “pair of images” refers to two images captured synchronouslyby a pair of cameras with an overlapping FOV. Alternatively oradditionally, an external computing device processes image data capturedby the camera pair.

The image store 305 is configured to store a pair of imagessynchronously captured by each of a pair of cameras. The synchronizationstore 310 is configured to store camera synchronization data. Examplesof synchronization data include time lags between cameras due to networklag or internal component lag (e.g., lag from the synchronizationinterface 220, the processor 202, the sensor controller 208, and thelike). The synchronization store 310 is configured to store calibrationsettings such as read window shift information and a calibrated time lagfor initiating image capture.

The capture controller 315 controls image capture by the image sensor206. In one embodiment, the capture controller 315 applies a calibrationcorrection to synchronize image capture with one or more additionalcameras. The calibration correction may include a read window shift by adetermined number of pixels, as determined by the pixel shiftdetermination module 320. The calibration correction may include,alternatively or additionally, a time lag for one of the cameras in thepair to delay relative to the other camera of the pair before beginningimage capture, as determined by the time lag determination module 330.

The pixel shift determination module 320 identifies a pixel shiftbetween an image captured by a first camera and an image captured by asecond camera. This pixel shift indicates spatial misalignment betweenthe image sensors of the cameras. In one embodiment, the pixel shiftdetermination module 320 determines a pixel shift in the rolling shutterdirection due to a misalignment between the image sensors along therolling shutter direction. The capture controller 315 can use thedetermined pixel shift to correct the misalignment between the imagesensors. Pixel shift determination is described in greater detail below.

The time lag determination module 330 determines a time lag between thecapture of an image row by a first camera and corresponding image row ofa second camera. The time lag determination module 330 can determine atime lag based on a pixel shift received from the pixel shiftdetermination module 320. Using the determined time lag, t_(lag), thecapture controller 315 synchronizes the pair of cameras by delayingimage capture of a first of the pair by the time lag relative to asecond of the pair. In one embodiment, an image sensor has an associatedrow time, t_(row), which represents an elapsed time between exposing afirst pixel row and a second, subsequent pixel row. If images taken by acamera pair are determined to have a pixel shift of n pixels, then thetime lag t_(lag) required to correct the pixel shift can be determinedusing the following equation:t _(lag) =t _(row) ×n

In one embodiment, calibrating image capture between cameras in a pairinvolves synchronously capturing images with the camera pair,determining a pixel shift between the captured images, and applying adetermined correction iteratively until the determined pixel shift isless than a pre-determined pixel shift threshold. The calibrationprocess may be initiated when cameras are powered on or paired, or auser may manually initiate the calibration process. A master camera caninitiate the calibration process when a time elapsed since the lastcalibration is greater than or equal to a pre-determined threshold. Inan embodiment with additional cameras, additional calibrations can beperformed among cameras having a common FOV.

Pixel Shift Determination

The pixel shift determination module 320 identifies a shift in pixelsbetween a pair of images, and includes an image reduction module 321, animage correlation module 322, and an edge correlation module 325, whichin turn includes an image smoothing module 326, an image cropping module327, an edge extraction module 328, and an edge matching module 329. Itshould be noted that the modules described in the embodiment of FIG. 3are just one means of implementing the functions described herein, andin some embodiments, different or additional modules perform similar oradditional functions. In addition, in some embodiments, certainfunctions are not performed. For example, in correlating edges between apair of images, the edge correlation module 325 may not smooth theimages or crop the images using the image smoothing module 326 and theimage cropping module 327, respectively.

The image reduction module 321 receives a pair of images and producesreduced image data for the pair of images, for instance by reducing anamount of image data from each captured image, from a portion of eachimage, from compressed image data representative of each image, and thelike. In some embodiments, the image reduction module 321 reduces imagedata by removing or compressing portions of image data that are lessuseful in identifying a pixel shift (e.g., image portions of uniformcolor, uniform texture, and the like). By reducing image data, the imagereduction module 321 can reduce the computational load required todetermine a pixel shift between a pair of images.

The reduced image data can be used by the pixel shift determinationmodule 320 to determine a pixel shift between an image pair, or can betransmitted to another device for use in a pixel shift determination. Inone embodiment, the image reduction module 321 selects a portion a pairof images representative of a common FOV of the pair of images andreduces image data of the pair of images within the common FOV, externalto the common FOV, or both. In some embodiments, the location of acommon FOV between an image pair is known, while in others, a common FOVcan be identified by analyzing the image pair to identify portions ofeach image in the pair that are substantially similar. The imagecompression module 321 may use known camera configurationcharacteristics (e.g., an angular width of the field of view, theorientation of the cameras, or the relative position of the cameras) toidentify portions of the image corresponding to an overlapping FOV.

The image reduction module 321 can apply one or more algorithms tocompress image data of a pair of images to reduce the image data of thepair of images. In one embodiment, the image reduction module 321compresses pixels in a direction perpendicular to the rolling shutterdirection of a pair of images to reduce image information, whilemaintaining image information along the direction of the rollingshutter. In one embodiment, the image reduction module downsamples imagedata of the pair of images to reduce image data, further decreasing thecomputational time required by the pixel shift determination module 320.The image can be downsampled by a downsampling rate, which reduces thetotal number of pixels along a row or column of pixels by acorresponding downsampling factor. It should be noted that in someembodiments not described further herein, the pixel shift determinationmodule 320 determines a pixel shift between a pair of images withoutreducing the image data of a pair of captured images.

The image correlation module 322 receives reduced image data for thepair of images and determines a pixel shift between the pair of imagesfrom the reduced image data. In one embodiment, the image correlationmodule 322 uses a cross-correlation algorithm to identify a pixel shiftbased on the reduced image data for the pair of images. In thisembodiment, a cross-correlation coefficient is calculated for thereduced image data of the pair of images for each of a plurality ofpossible pixel shifts along one or more dimensions of the image data.The image correlation module 322 can determine the pixel shift betweenthe pair of images as the pixel shift that results in the highestcross-correlation coefficient. The image correlation module 322 maycalculate a cross-correlation coefficient for all possible pixel shiftsor may use one or more correlation optimization methods to search for apixel shift that maximizes a cross-correlation coefficient whilereducing or substantially minimizing the computational load required toidentify a pixel shift between an image pair. The image correlationmodule 322 can determine image vectors between features of each of thepair of images for use in determining a pixel shift. For example, theimage correlation module 322 can identify features of the image datasuch as corners (e.g., using the Harris and Stephens corner detectionalgorithm or the Förstner corner detector) or blobs (e.g., using aLaplacian analysis or by determining a difference between Gaussianscurves representative of the images). Vectors between correspondingimage features can then be determined (e.g., using a correlationalgorithm) to determine a pixel shift. It should be noted that inembodiments where the pixel shift determination module 320 does notreduce image data, the image correlation module can identify a pixelshift between the original pair of images using the methods describedherein.

The edge correlation module 325 detects one or more edges in a pair ofimages or in reduced image data associated with a pair of images, andcorrelates the detected edges to determine a pixel shift. In theembodiment of FIG. 3, the edge correlation module 325 includes an imagesmoothing module 326, an image cropping module 327, an edge extractionmodule 328, and an edge matching module 329 for use in correlating edgesto determine a pixel shift. The edge correlation module 325 can useCanny edge detection or differential edge detection to identify edges inthe image pair. In some embodiments, the edge correlation module 325identifies edges using luma and/or chroma information from a pair ofimages in the YUV color space. In such embodiments, the edge correlationmodule 325 identifies image regions to search for edges based on chromadata of the image pair, and detects edges within the identified regionsusing luma data of the image pair. In some embodiments, the edgecorrelation module 325 multiplies a derivative of a chroma component ofan image by a derivative of a luma component of the image to detectedges within the image. In such embodiments, edges can be identifiedbased on pixel locations associated with an above-threshold product ofsuch derivatives.

The image smoothing module 326 removes artifacts and otherirregularities from the image pair or from reduced image data associatedwith the image pair, producing smoothed images. In one embodiment,Gaussian filters are used to identify edges and to remove imageartifacts that are determined to not be edges. For example, a smallGaussian filter can be used to detect fainter, less obvious edges, and alarge Gaussian filter can be used to detect more prominent edges. Insuch an example, the image smoothing module 326 can remove artifacts notdetected as edges by either filter. The image smoothing module 326 mayalso downsample an image pair to reduce processing time and/or to smooththe image pair.

The image cropping module 327 removes portions of the smoothed pair ofimages to produce a pair of cropped images for use in detecting edgeswithin the images. Alternatively or additionally, the image croppingmodule 327 removes portions of reduced image data to produce a pair ofcropped and reduced image data. In one embodiment, the edge correlationmodule 325 identifies straight-line edges perpendicular to the rollingshutter direction. Some lenses create a fisheye effect in capturedimages, which distorts straight lines. This fisheye effect is morepronounced around the edges of an image. To reduce or remove a fisheyeeffect in captured images, the image cropping module 327 crops theimages by removing image data from the outer borders of the image toreduce such warping of straight-line edges.

The edge extraction module 328 identifies edges in the pair of croppedimages or in reduced image data associated with the pair of croppedimages, for instance using any of the techniques described herein, andextracts edge information representative of the edges of the pair ofimages. In one embodiment, the edge extraction module 328 extracts edgemagnitude and edge phase for each detected edge in the pair of images.To calculate edge magnitude and phase, first derivatives are taken forthe cropped image pair along the rows and columns of pixels. Forexample, the edge extraction module 328 can take the first derivative ofimage luma data, image chroma data, red/green/blue data, black and whitedata, or any other data representative of the image pair. The edgemagnitude is computed based on a square root of the squared sums of thefirst derivatives of the cropped image pair along the row and columndirections. The edge phase is computed based on an inverse tangent ofthe ratio of the first derivatives of the cropped image pair along therow and column directions. The edge phase corresponds to an edge'sorientation, and the edge magnitude corresponds to an edge's sharpness.It should be noted that in other embodiments, the edge extraction module328 extracts edge information other than edge magnitude and phaseinformation.

In one embodiment, the edge extraction module 328 identifiessubstantially straight edges having a direction substantiallyperpendicular to the rolling shutter direction. For example, if therolling shutter direction is vertical, the edge extraction module 328identifies horizontal edges. The edge extraction module 328 can identifypixels in an identified edge as a component of the edge if thedetermined edge magnitude is greater than or equal to a magnitudethreshold, and if a difference between the determined edge phase and atarget edge phase is within an edge threshold value. In one embodiment,the edge extraction module 328 determines a length of each identifiededge, and discards edges having a length smaller than a lengththreshold.

The edge matching module 329 accesses the edge informationrepresentative of the edges in the pair of images (or the reduced dataassociated therewith) and determines a pixel shift based on the edgeinformation. In one embodiment, the edge matching module 329characterizes edges represented by the edge information by their length,their magnitude, their phase, or their position along the axis of therolling shutter. In one embodiment, the edge matching module 329recursively matches edges based at least in part on edge length and/oredge magnitude. As used herein, “matching edges” refers to the alignmentof corresponding edges in the pair of images. To recursively matchedges, an edge from a first image is compared against edges from asecond image until a match occurs within an edge length differencethreshold and/or an edge magnitude difference threshold. The edgematching module 329 can determine a pixel shift between the pair ofimages that maximizes the number of matched edges or maximizes acorrelation score representative of the alignment between correspondingedges. The edge matching module 329 may score possible pixel shifts onthe basis of correlation between edges in the image pair, or may use adirected search based on matching the longest edges within the imagepair. It should be noted that if the image reduction module 321downsampled the pair of images by a downsampling factor before the edgecorrelation module 325 accesses the pair of images to determine a pixelshift, then the pixel shift determined by the edge correlation moduleshould be multiplied by the downsampling factor prior to using the pixelshift to align image sensors.

Image Sensor Alignment

FIG. 4A illustrates the alignment of an example pair of image sensors byshifting the read window on an image sensor of the pair, according toone embodiment. Example image sensors 410A and 410B are taken by camerashaving a different alignment in the rolling shutter direction. Eachimage sensor 410 includes a read window 420, which indicates the pixelsin the image sensor 410 that are exposed to light and read out by thecorresponding camera. In the illustrated example, the image sensors 410capture image data from the top row of pixels of the read window 420 tothe bottom row of pixels of the read window 420. The image sensors 410capture light incident from a partially overlapping FOV, including theoverlap region 425. The image sensors 410 also capture light incidentfrom non-overlapping FOVs, including the regions 423A and 423B.

In the illustrated example, the read windows 420 are adjusted tosynchronize image capture by an amount of pixels substantiallyequivalent to a determined pixel shift (for instance, from the pixelshift determination module 320). As aligned, image sensor 410A captureslight incident from a higher FOV relative to the image sensor 410B. Theread windows 420A and 420B illustrate the initial position of the readwindows prior to adjustment. Due to the read window alignment, the imagesensor 410A captures a row of pixels at an earlier time than acorresponding row of pixels having a similar height in image sensor410B. In other words, the read window 420B is positioned below the readwindow 420A, resulting in a misalignment between the image sensors.Since the image sensors 410 begin capturing images at the top of theread window, shifting the read window 410B upwards by the determinedpixel shift can synchronize the time of image capture of correspondingrows of pixels between the image sensors. The capture controller 315shifts the initial read window 420B to the position of the aligned readwindow 420C (as shown). When images are captured, corresponding pixelrows from images captured by the image sensors 410 contain image datacaptured at substantially the same time, and can be stitched together toconstruct a 2D image. In addition, after shifting a read window by adetermined pixel shift, the image regions 425A and 425B represent thesame FOV, and can be used to construct a 3D image.

FIG. 4B illustrates the alignment of an example pair of image sensors410 using a time lag, according to one embodiment. Each image sensor 410includes a read window 420, which indicates the pixels in the imagesensor 410 that are exposed to light and read out by the correspondingcamera. In the illustrated example, the image sensors 410 capture imagedata from the top row of pixels of the read window 420 to the bottom rowof pixels of the read window 420. The image sensors 410 capture lightincident from a partially overlapping FOV, including the overlap region425. The image sensors 410 also capture light incident fromnon-overlapping FOVs, including the regions 423, 427, and 429.

In the illustrated example, a delay time is applied to synchronize imagecapture according to a determined time lag (for instance, from the timelag determination module 330). As aligned, image sensors 410A and 410Bhave a different alignment in the rolling shutter direction. Due to theread window alignment, the image sensor 410B captures a row of pixels atan earlier time than a corresponding row of pixels having a similarheight in image sensor 410A. To synchronize the image sensors 410, thecapture controller 315 applies a time lag from the time lagdetermination module 330. The capture controller 315 initiates imagecapture by image sensor 410A. During the applied time lag, the imagesensor 410A captures the pixels in regions 427A. After the passage oftime equivalent to the time lag, the capture controller 315 initiatesimage capture by image sensor 410B, so the image sensors 410 capturecorresponding rows of pixels in the image areas 423 and 425 atsubstantially the same time. The regions 423 and 425 from each imagesensor can be stitched together to construct a 2D image, and the regions425A and 425B, representing a common field of view, can be combined tocreate a 3D image. Further, the regions 427 and 429 contain pixel rowscontaining image data that do not correspond to a similar height in thecaptured image. If images 410A and 410B are combined into a rectangular2D image, then the image data from regions 427 and 429 may be discarded.

Horizontal Edge Detection

FIGS. 5A-5D illustrate example images illustrating edge detection usedfor determining a pixel shift, according to one embodiment. Image 500Aillustrates an image containing several substantially horizontal edges(e.g., window edges, window ledges). Image 500B illustrates edgemagnitude information determined for the image 500A. The edge magnitudeinformation can be, for instance, determined by the edge correlationmodule 325. In image 500B, image data having a relatively high edgemagnitude is shown in white, and image data having a relatively low edgemagnitude is shown in black. Image 500C illustrates edge phasedetermined for the image data from image 500A, for instance asdetermined by the edge correlation module 325. In image 500C, thedirection of an edge corresponds to the color of that edge. Edges can beidentified by analyzing neighboring pixels having a substantiallysimilar color. Regions having no apparent edges appear to have randomcolor.

Image 500D illustrates horizontal edges identified from the image dataof image 500A using the edge magnitude data illustrated in image 500Band the edge phase data illustrated in image 500C. Image datacorresponding to substantially horizontal edges are illustrated inwhite, and other image data are illustrated in black. The brightness orthickness of the identified horizontal edges corresponds to anidentified edge's edge magnitude data. Some possible horizontal edgeshave been discarded in image 500D, for instance because the edge lengthfor such edges is less than a length threshold.

In some embodiments, the edge extraction described herein can be appliedto a calibration target used to align a pair of cameras. The calibrationtarget is placed in the overlapping FOV of the cameras, and the camerascapture a pair of images of the calibration target to determine a pixelshift between the image sensors of the cameras. The calibration targetcan be any object suitable for determining a pixel shift between theimage sensors of the cameras. Example calibration targets include aQuick Response or “QR” code, a Macbeth chart, or any other suitablemarker. When used with the pixel shift determination methods describedherein, a calibration target's edges may be aligned to be substantiallyperpendicular to the rolling shutter direction associated with the imagesensors of the camera pair to improve pixel shift determinationperformance.

Aligning Image Sensors in a Multi-Camera System

FIG. 6A illustrates a flow diagram of a process for aligning imagesensors in a multi-camera system, according to one embodiment. A pair ofimage sensors captures 610 a pair of images at substantially a sametime. The image sensors have an overlapping FOV, resulting in a portionof each captured image representing the overlapping FOV.

A pixel shift between the captured images is identified 620 based onimage data of the pair of images representative of the common FOV.Identifying the pixel shift can include correlating the image datarepresentative of the overlapping FOV. For example, to correlate theimage data, at least one image is shifted by a number of pixels so thata measure of the difference between the pixel information of theportions of the images representative of the overlapping FOV issubstantially reduced or minimized. The number of pixels used to shiftthe image that results in the measure of the difference between pixelinformation being substantially reduced or minimized is the identifiedpixel shift. Before correlating the captured image data, the image dataof the images can be compressed and/or reduced, and the correlation canbe determined based on the compressed or reduced image data. If theportions of the images representative of the overlapping FOV include abelow-threshold amount of image data, one or more additional pairs ofimages having an overlapping FOV can be captured before identifying apixel shift.

A time lag between the image sensors is determined 630 based on theidentified pixel shift, for instance based on an amount of time requiredby an image sensor to capture a number of pixel lines corresponding tothe pixel shift. At least one of the image sensors is calibrated 640 tosubstantially synchronize subsequent image capture by the image sensors.For example, the determined time lag is applied to at least one of theimage sensors. If a first image sensor of the pair of image sensors lagsa second image sensor of the pair of image sensors during image capture,then the first image sensor can initiate image capture, and the secondimage sensor can begin image capture after waiting for a period of timeequal to the determined time lag. Alternatively or additionally, theread window of at least one of the image sensors can be adjusted by anumber of pixels equal to the identified pixel shift, for instance inthe rolling shutter direction or opposite the rolling shutter direction.For example, if a pixel shift of 3 pixel rows is identified, and a firstimage sensor lags a second image sensor, the first image sensor canadjust the read window of the first image sensor downward by 3 pixelrows such that the first image sensor captures a pixel row correspondingto a pixel row of the second image sensor at substantially the sametime.

FIG. 6B illustrates a flow diagram of a process for determining a pixelshift between images having an overlapping FOV, according to oneembodiment. A pair of images, including a first image and a secondimage, is captured at substantially the same time. The images includeimage data representative of an overlapping FOV. The portions of theimages representing the overlapping FOV are determined 621 between thepair of images. The images, or the portions thereof representing theoverlapping FOV, are compressed 622, for example by a downsamplingfactor. The portions of the images representing the overlapping FOV canalso be cropped to remove image portions determined to be distorted dueto a fisheye effect. The compressed images are smoothed 623, for exampleby convolving a Gaussian blur filter with the compressed image portions.

Edges within the smoothed image portions are extracted 624, for instanceby determining edge magnitude information and edge phase informationbased on the luma components of the smoothed image portions. Theextracted edges are matched 625, and a pixel shift is determined 626 forthe pair of images based on the matched edges. As noted herein, thedetermined pixel shift can be used to align a pair of image sensors, ora time lag can be determined based on the pixel shift, and the time lagcan be used to align the image sensors.

Additional Configuration Considerations

The disclosed embodiments beneficially allow for paired image sensors tocalibrate themselves without manual calibration. By capturing the pairof images and determining a pixel shift between the images, the camerascan determine a calibration factor between them. Using a read windowshift and/or a time lag on image capture, the cameras can use thedetermined pixel shift to correct the positional misalignment and/orlack of temporal synchronization between image sensors. As a result,image sensors capture image data of moving objects suitable for use in acombined 2D or 3D image without calibrating the camera's position byhand.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms, for example, as illustrated inFIGS. 2 and 3. Modules may constitute either software modules (e.g.,code embodied on a machine-readable medium or in a transmission signal)or hardware modules. A hardware module is tangible unit capable ofperforming certain operations and may be configured or arranged in acertain manner. In example embodiments, one or more computer systems(e.g., a standalone, client or server computer system) or one or morehardware modules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

The performance of certain of the operations may be distributed amongthe one or more processors, not only residing within a single machine,but deployed across a number of machines. In some example embodiments,the one or more processors or processor-implemented modules may belocated in a single geographic location (e.g., within a homeenvironment, an office environment, or a server farm). In other exampleembodiments, the one or more processors or processor-implemented modulesmay be distributed across a number of geographic locations.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context.Further, unless expressly stated to the contrary, “or” refers to aninclusive or and not to an exclusive or.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs for asystem and a process for synchronizing multiple image sensors throughthe disclosed principles herein. Thus, while particular embodiments andapplications have been illustrated and described, it is to be understoodthat the disclosed embodiments are not limited to the preciseconstruction and components disclosed herein. Various apparentmodifications, changes and variations may be made in the arrangement,operation and details of the method and apparatus disclosed hereinwithout departing from the spirit and scope defined in the appendedclaims.

What is claimed is:
 1. A computer-implemented method for determining apixel shift between an image pair captured by image sensors, the methodcomprising: accessing a first image and a second image of the imagepair, the first and second images comprising image data representativeof an overlapping field of view between the image sensors; performing asmoothing operation on the first image and the second image to producesmoothed image data; determining edge magnitude data and edge phase datafrom the smoothed image data; identifying, by one or more processors,one or more edges in the image data based at least in part on the edgemagnitude data and the edge phase data of the smoothed image data; anddetermining a pixel shift between the image pair by matching a firstedge in the first image with a second edge in the second image usingdetermined edge lengths for the identified one or more edges.
 2. Themethod of claim 1, wherein identifying the one or more edges in theimage data comprises: identifying a first set of smoothed image datahaving determined edge magnitude greater than an edge magnitudethreshold; identifying a second set of smoothed image data havingdetermined edge phase within a threshold of a target edge phase value;and identifying the one or more edges from image data in both the firstset of smoothed image data and the second set of smoothed image data. 3.The method of claim 2, wherein the identified one or more edges aresubstantially perpendicular to a rolling shutter direction of the imagesensors, and the target edge phase value corresponds to the rollingshutter direction.
 4. The method of claim 1, wherein the image sensorshave a rolling shutter direction, the method further comprising: beforedetermining the edge magnitude data and edge phase data, compressing thefirst and second images perpendicular to the rolling shutter directionwithout compressing the first and second images along the rollingshutter direction.
 5. The method of claim 1, wherein the first image andthe second image are downsampled before the smoothing operation isperformed.
 6. The method of claim 1, further comprising cropping thefirst and second images by removing image data corresponding to outerborders of the first and second images prior to determining the edgemagnitude data and the edge phase data.
 7. The method of claim 1,further comprising calibrating the image sensors using the determinedpixel shift.
 8. A non-transitory computer readable medium comprisinginstructions for determining a pixel shift between an image paircaptured by image sensors, the instructions executable by a processor,the instructions comprising instructions for: accessing a first imageand a second image of the image pair, the first and second imagescomprising image data representative of an overlapping field of viewbetween the image sensors; performing a smoothing operation on the firstimage and the second image to produce smoothed image data; determiningedge magnitude data and edge phase data from the smoothed image data;identifying, by one or more processors, one or more edges in the imagedata based at least in part on the edge magnitude data and the edgephase data of the smoothed image data; and determining a pixel shiftbetween the image pair by matching a first edge in the first image witha second edge in the second image using determined edge lengths for theidentified one or more edges.
 9. The computer-readable medium of claim8, wherein identifying the one or more edges in the image datacomprises: identifying a first set of smoothed image data havingdetermined edge magnitude greater than an edge magnitude threshold;identifying a second set of smoothed image data having determined edgephase within a threshold of a target edge phase value; and identifyingthe one or more edges from image data in both the first set of smoothedimage data and the second set of smoothed image data.
 10. Thecomputer-readable medium of claim 9, wherein the identified one or moreedges are substantially perpendicular to a rolling shutter direction ofthe image sensors, and the target edge phase value corresponds to therolling shutter direction.
 11. The computer-readable medium of claim 8,wherein the image sensors have a rolling shutter direction, wherein theinstructions further comprise instructions for: before determining theedge magnitude data and the edge phase data, compressing the first andsecond images perpendicular to the rolling shutter direction withoutcompressing the first and second images along the rolling shutterdirection.
 12. The computer-readable medium of claim 8, wherein theinstructions further comprise instructions for calibrating the imagesensors using the determined pixel shift.
 13. A camera system comprisingat least one processor, an image sensor, and a non-transitory computerreadable medium comprising instructions for determining a pixel shiftbetween an image pair captured by image sensors including the imagesensor, the instructions executable by the at least one processor, theinstructions comprising instructions for: accessing a first image and asecond image of the image pair, the first and second images comprisingimage data representative of an overlapping field of view between theimage sensors; performing a smoothing operation on the first image andthe second image to produce smoothed image data; determining edgemagnitude data and edge phase data from the smoothed image data;identifying, by one or more processors, one or more edges in the imagedata based at least in part on the edge magnitude data and the edgephase data of the smoothed image data; determining a pixel shift betweenthe image pair by matching a first edge in the first image with a secondedge in the second image using determined edge lengths for theidentified one or more edges.
 14. The camera system of claim 13, whereinidentifying the one or more edges in the image data comprises:identifying a first set of smoothed image data having determined edgemagnitude greater than an edge magnitude threshold; identifying a secondset of smoothed image data having determined edge phase within athreshold of a target edge phase value; and identifying the one or moreedges from image data in both the first set of smoothed image data andthe second set of smoothed image data.
 15. The camera system of claim14, wherein the identified one or more edges are substantiallyperpendicular to a rolling shutter direction of the image sensors, andthe target edge phase value corresponds to the rolling shutterdirection.
 16. The camera system of claim 13, wherein the image sensorshave a rolling shutter direction, wherein the instructions furthercomprise instructions for: before determining the edge magnitude dataand the edge phase data, compressing the first and second imagesperpendicular to the rolling shutter direction without compressing thefirst and second images along the rolling shutter direction.
 17. Thecamera system of claim 13, wherein the instructions further compriseinstructions for calibrating the image sensors using the determinedpixel shift.